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GRANT  # :  N00014-97-10396 

PRINCIPAL  INVESTIGATOR:  Dr.  Norman  I.  Badler 
INSTITUTION:  University  of  Pennsylvania 

GRANT  TITLE:  Hand  Tool  Manipulation  and  Self-Presence  in  VR 


AWARD  PERIOD:  1  April  1997  -  30  September  1998 

OBJECTIVE:  To  purchase  Virtual  Reality  and  associated 

computing  equipment  to  enable  the  investigation  of  full 
body  avatar  control  and  self-presence  using  both  fine 
motion  hand  manipulation  plus  force  feedback  as  well  as 
whole  body  posture  and  motion  sensors. 

APPROACH:  Creating  whole  body  avatars  requires  software  to 
model  and  move  the  virtual  human  body  representation  of 
self.  We  already  had  in  house  Jack  software  that  served 
this  purpose.  But  we  had  no  modern  VR  instrumentation  that 
would  be  able  to  effectively  sense  a  live  participant's 
hand  and  body  attitude,  and  provide  this  information  to  the 
Jack  software  in  a  usable  format.  New  VR  tools  and 
computing  power  were  added  in  our  laboratory  through  this 
grant  to  enable  real-time  human  motion  capture  and  hand 
force  feedback. 

ACCOMPLISHMENTS:  The  purchased  equipment  has  been  an 

integral  part  of  the  Center  for  Human  Modeling  and 
Simulation's  laboratory  facilities.  It  has  been  used  to 
study  human  gestures,  the  "feel"  of  small  hand  tools  and 
instruments,  and  motion  capture  mapping  across  different 
sized-individuals . 

•  In  studying  human  gestures,  the  equipment  was  used  to 
capture  the  gestural  and  postural  motions  of  dancers, 
movement  notators,  music  conductors,  and  American  Sign 
Language  signers.  Position,  velocity,  acceleration,  and 
jerk  (3rd  derivative)  curves  .were  obtained  from  live 
motion  capture  using  the  VR  sensors  and  the  computers. 
From  these  observations,  a  kinematic  theory  and 
implementation  of  gestural  Effort  was  proposed  in  Diane 
Chi's  PhD  Dissertation.  This  system  uses  just  four 
parameters  —  derived  from  Laban's  Efforts  to 
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qualitatively  change  the  performance  of  given  arm 
gestures  of  an  animated  character. 

•  To  understand  the  "feel"  of  small  handtools,  a 
demonstration  system  was  constructed  by  Drew  Kessler  and 
undergraduate  Mark  Palatucci.  They  graphically  modeled  a 
small  pistol  and  its  trigger,  then  developed  the  force- 
feedback  structure  that  drove  the  CyberGrasp  glove.  The 
user  wearing  the  glove  could  feel  the  pistol  and  the 
grip,  and  could  feel  the  movement  and  resistance  of  the 
trigger.  By  aiming  at  a  simulated  target,  the  user  could 
experience  a  virtual  shooting  range. 

•  Motion  capture  for  whole  body  avatars  is  complicated  by 

the  typical  requirement  that  the  graphical  model  have  the 
exact  same  articulation  and  body  segment  (joint-to- joint 
dimensions)  as  the  user.  The  calibration  of  the  motion 
sensing  device  to  the  individual  user  is  tricky  at  best 
and  always  awkward.  Errors  in  matching  the  model  to  the 
user  result  in  captured  motions  that  relate  improperly  to 
objects  in  the  environment  or  features  on  the  user's, 
self:  for  example,  accurate  reach  to  grasp  and  move  a 
glass  from  table  to  mouth  is  difficult  or  impossible. 
The  problem  is  that  the  joint  angles  of  the  user  are  not 
matching  the  joint  angles  of  the  model.  Likewise, 
features  (such  as  the  mouth)  may  be  at  different 
locations  in  space.  Our  approach  is  to  use  semantic 
features  on  the  model  and  acceleration  changes  (zero- 
crossings)  to  determine  spatial  adjacencies  of  limb  and 
object  that  appear  important.  These  adjacencies  are  used 
to  drive  an  inverse  kinematics  algorithm  that 

automatically  adjusts  for  the  differences  in  segment 
length.  Additionally,  the  motion  style  (velocity 

profile)  along  the  movement  is  scaled  a  preserved.  As  a 
demonstration  of  the  approach,  an  adult  user  controlled 
the  motions  of  a  child-sized  model  drinking  from  a  glass. 
The  eye  gaze  of  the  child  model  was  also  corrected  for 
the  spatial  mismatch  in  sizes  and  feature  locations. 
This  work  by  Rama  Bindiganvale,  was  reported  at  the 
Motion  Capture  Workshop  in  Geneva,  Switzerland,  in 
November  1998. 

CONCLUSIONS :  Our  work  with  the  VR  sensors  and  force 
feedback  has  led  directly  to  one  PhD  (Chi)  and  one 
published  conference  paper  (Bindiganavale  and  Badler) .  The 
preliminary  work  by  Bindiganavale  will  be  incorporated  into 
her  PhD  Dissertation  as  well.  The  VR  target  range  was 
documented  in  a  videotape  by  Palatucci  and  Kessler.  We 


were  able  to  do  this  research  as  3  direct  consequence  of 
the  new  VR  and  computing  equipment  purchased  on  this  DURIP 
grant . 

SIGNIFICANCE:  We  have  a  much  better  understanding  of  how 
live  user  input  can  be  used  to  drive  a  3D  articulated  human 
model  of  a  different  size.  We  also  have  a  parameterized 
model  of  human  gesture  based  on  Laban  Efforts.  This  model 
is  the  first  implementation  and  computational  evaluation  of 
the  Laban  Effort  system,  a  widely  used  notational  system 
for  qualitative  human  movements. 

PATENT  INFORMATION:  There  are  no  patents  filed  based  on 
this  work,  though  the  feasibility  of  a  patent  on  Chi's 
gesture  work  was  investigated. 

AWARD  INFORMATION:  Diane  Chi  received  her  PhD  in  Computer 
and  Information  Science  in  1999. 
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